1. Field of the Invention
The invention relates generally to input devices, and more particularly, to improving handling and battery life of a mouse which can also be used in-air.
2. Description of the Related Art
The personal computer (PC) is increasingly becoming a media center, which users use to store and play music, videos, pictures, etc. As a result, the PC is also increasingly seen in the living room. Users are often in more relaxed positions (e.g., lounging on a couch) when in the living room, or more generally, when interacting with media (such as when watching a video), than when using the PC for more traditional interactions. Apart from being in a more relaxed position, the user is often not close enough to a desk to rest a mouse on it.
Use of pointing devices such as mice or trackballs with a PC has become ubiquitous. At least for the reasons discussed above, such input devices which can only function when placed on a work surface (e.g., a desk or a mouse pad) are not optimal for use in such environments. Some attempts have been made at creating input devices which work freely in the air, and also work as more traditional input devices when placed on a surface. The most intuitive interface is to map the system orientation changes (e.g., yaw and pitch) into x and y cursor displacements. FIG. 1, from Logitech U.S. Pat. No. 6,069,594, illustrates yaw, pitch and roll. Some available “in-air” devices measure the changes in these orientations (yaw, pitch, and/or roll) when the user moves the device, and use these to change the position of a cursor appearing on a computer screen or media player screen. For example, the cursor on the screen moves by an amount that is a function of the yaw and pitch change. In its simplest form, the cursor position change is proportional to the orientation angle change, for example 20 pixels cursor movement results from a 1° angle change or increment. In some available devices, yaw controls the x-coordinate and pitch controls the y-coordinate. More elaborate methods, not described here, apply some non linear function on the estimated yaw, pitch, and/or roll.
Several patents and publications describe detection of movement in 3D and/or detection of movement in air, and using this detected movement to control cursor movement on an associated display. U.S. Pat. No. 5,543,758 describes a remote control that operates by detecting movement of the remote control in space including detecting circular motions and the like. U.S. Pat. No. 6,104,380 describes a control device for controlling the position of a pointer on a display based on motion detected by a movement sensor. U.S. Pat. No. 5,554,980 describes a mouse that detects 3D movement for controlling a cursor on a display. U.S. Pat. No. 5,363,120 claims a system and a method for a computer input device configured to sense angular orientation about a vertical axis. The detected orientation is used to control a cursor position on a screen. U.S. Pat. No. 4,578,674 shows a wireless (ultrasonic) pointer that can also be operated in 3 dimensions. Also, U.S. Pat. No. 4,796,019 shows a wireless handheld pointer to control a cursor by changing angular position using multiple radiation beams. IBM Technical Disclosure Bulletin Vol. 34, No. 11 describes a Gyroscopic Mouse Device that includes a gyroscope that is configured to detect any movement of a mouse to control a cursor on a display. U.S. Pat. No. 5,898,421 describes a gyroscopic mouse method that includes sensing an inertial response associated with mouse movement in 3D-space. U.S. Pat. No. 5,440,326 describes a gyroscopic mouse configured to detect mouse movement in 3D-space, such as pitch and yaw. U.S. Pat. No. 5,825,350 describes a gyroscopic mouse configured to detect mouse movement in 3D-space. U.S. Pat. No. 5,448,261 describes a mouse configured to move in 3D space. U.S. Pat. No. 5,963,145, U.S. Pat. No. 6,147,677, and U.S. Pat. No. 6,721,831 also discuss remote control orientation. U.S. Pat. No. 6,069,594 shows a mouse that moves in 3 dimensions with 3 ultrasonic, triangulating sensors around the display. U.S. Published Application 20050078087 is directed to a device which acts as a mouse for a PC when on a surface, detects when it is lifted, then acts as a remote control for appliances. U.S. Published Application 20022040095317 also discloses a remote control that can be used to control a television and a computer system.
However, known pointing devices that work “in-air” present several challenges. For instance, clicking at a specific desired location (e.g., on an icon on a display associated with the computer) is difficult when maneuvering the pointing device in air. Further, since such devices work both when resting on a surface (e.g., like a conventional mouse) as well as in air there needs to be some way to figure out whether the device is operating on a surface or in air. Moreover, a device which operates in air may be subject to significant ongoing battery consumption in certain scenarios, such as if the user were to continually hold the device in his hand while watching a movie. Furthermore, easy and intuitive user interfaces are not available in in-air input devices for computers, such as for controlling volume and so on. Each of these is discussed in some detail below.
A fundamental requirement of a pointing device is the point-and-click mechanism—for example, clicking on an icon on a display associated with the computer. The cursor is moved until it reaches the icon and then the user clicks upon it to trigger some action. Other functionality typically associated with such pointing devices includes drag-and-drop. Here an object is selected by clicking a button on the pointing device while the cursor is positioned on the object, moved while the button is maintained pushed, and then dropped by releasing the button when the destination has been reached. Clicking at a precise location with an in-air device is problematic. To begin with, controlling the movement of the cursor accurately is difficult because holding the device in the air makes complete control of orientations difficult. Further, changes in the device orientation and/or location will, by design move the cursor, and so will the parasitic motion generated by the hand when clicking. Typically, such parasitic motion results in a missed click (the operating system suppresses click when the cursor is not steady over its target or when the cursor is no longer on the icon at button release). Moreover, there is an inherent trade-off in free-space devices—given the limited angle span of a human wrist, a large resolution is needed in order for the user to easily reach the entire screen with a single wrist movement; however having too large a resolution would result in random cursor movements attributable to normal tremor of human hands and parasitic clicking motion.
Moreover, as mentioned above, there needs to be some way for the pointing device to know whether it is working in air, or on a surface, since cursor control is generally implemented differently in these two modes.
Some existing solutions, such as the one from Thomson's Gyration Inc. (Saratoga, Calif.), require the user's to take a specific action each time when he wants the device to operate in air. For example, a specific button (trigger) may be used, the state of which can indicate whether to make the in-air tracking mechanism active. However, such solutions are cumbersome to the user, since they require continual added actions (pressing and releasing the trigger button) from the user. The user cannot just pick up the device from the table and seamlessly continue to use it.
The solutions mentioned above address the problem of clicking in a specific location by simply exiting the in-air cursor control mode when clicking any buttons. When a user wants to click in a specific location, he simply releases the trigger button mentioned above, so that the movement of the device in air no longer translates into cursor movement. He then clicks on the button, thus eliminating any parasitic motion problems. The user will then have to click on the trigger button again to enter in-air cursor control mode to continue moving the cursor in air. Such pre-existing solutions result in a cumbersome, complicated and non-intuitive interaction of the user with the user interface.
Existing devices also suffer from the reduced battery life that results from moving the device unintentionally over an extended period of time, for example by holding the device while watching a movie or listening to music. Once again, existing system address this problem by requiring a trigger button to be pressed for the device to enter the in-air mode. When the trigger button is not pressed battery power is not consumed even if the device is moved around unintentionally in air. However, such a trigger button makes the system less intuitive to use.
Easy and intuitive interactions with the user interface (UI) on the display associated with the computer are very important. In the case of pointing devices that operate only on a surface, some user interfaces exist which permit such interactions.
Some such user interfaces exist even in the case of devices operating in air. U.S. Pat. No. 6,603,420 describes using movement of a remote control to control volume or the channel. This patent describes movement of the remote control device in a first plane controlling a first controlled parameter, e.g., receiver volume, while movement of the remote control device in a second plane perpendicular to the first plane controls a second parameter, e.g., channel selection. U.S. Pat. No. 6,750,801, discloses a tilt sensor in a remote control by which, depending on the angle and orientation of the tilt, different electronic devices can be selected for control (for example, tilt left selects the TV, tilt right selects the VCR). U.S. Pat. No. 4,796,019 discusses basic gesture of moving up and down, side to side, and tilting. U.S. Pat. No. 5,448,261 discusses using the position of a remote control to control cursor position. To change channel, the cursor is put on a channel number in a two dimensional grid. To change volume, the cursor is placed on a volume bar and moved. U.S. Publication No. 2004-0090423 describes a remote control which has a pointing device (e.g., touchpad, trackball, joystick) which can detect movement up and down on the pointing device to select menu items on a screen, and side to side to select a value for an item. However, none of these patents or publications provides an air-scroller type of mechanism. Further, none of these patents or publications provides the user with feedback regarding the desired action being taken, without the user attentively looking to the screen until the desired position is reached.
US Published Application 2004-218104 describes a remote control with a user interface which permits the user to select channels and quickly navigate a dense menu of options. As mentioned above, this publication also does not discuss providing the user with feedback regarding the desired action being taken, without the user attentively looking to the screen until the desired position is reached. Given the device is also used for media application, where the user is in a lean-back posture, precise positioning using visual feedback is cumbersome. This applies to the user's interaction with sliders, list boxes, pop-up menus, button groups, etc. as well.
There is thus a need for an easy to use pointing device that can seamlessly transition from working on a surface to working in free-space. Further, there is need for an in-air pointing device which can facilitate clicking at a desired location in an easy and intuitive manner. Moreover, there is a need for a non-cumbersome in-air pointing device which conserves battery life. Furthermore, there is need for an in-air pointing device with improved user interface interaction.